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1 26.10.2000 

Processing of color signals in a color camera 

The invention relates to a method and device for processing color signals in a 
color camera, and to a color camera comprising such a device. 



In a color camera, white balance is achieved when a neutral white object 
5 image by the camera under given illumination, is represented as red, green and blue signals 
having equal output levels. White balance is needed since the RGB representation produced 
by a color camera typically changes as the illumination of a scene varies. In some 
circumstances, a color camera white balance for certain illumination conditions, will not be 
white balanced for other illumination conditions. As a result, it is possible that an object 

10 under two different illuminations will have two different RGB representations even though a 
human observer would perceive the object as having the same color under both illuminations. 
In a known camera an image is captured by an RGB sensor. The output of this sensor 
contains basic analog color signals wherein each of the signals corresponds to a gain in one 
of the basic colors. These analog color signals are subsequently converted into at least one 

IS digital signal. The at least one digital signal is further subjected to a reconstruction method 
for obtaining reconstructed signals. The reconstructed signals comprise a first basic color 
signal R, a second basic color signal G and a third basic color signal B. These reconstructed 
signals are then further subjected to a correction method for obtaining standardized signals, 
suitable for worldwide acceptance. This correction method comprises of a multiplication of a 

20 three color signal matrix containing the first, second and third basic color signal R, G, and B 
by a correction matrix containing nine adjustable coefficients. In a usual color camera the 
white balance method is then applied to the standardized signals. 

A disadvantage of this method is that quantization errors occurring during the 
A/D conversion step remain present in at least one of the standardized signals. 

25 

The object of the invention is to meet this problem and to minimize the 
quantization errors in the fully processed signals. To this end, the invention provides a 
processing method and device, and a color camera as defined by the independent claims. The 
dependent claims define advantageous embodiments. 
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2 26.10.2000 
Preferably, the invention provides a method and device for processing analog 

color signals, comprising: 

analog preprocessing sensor output signals to obtain analog preprocessed 
signals that cause a reduced amount of digital quantization errors; 
5 converting the analog preprocessed signals into digital signals; 

reconstructing a first basic color signal ( /? ), a second basic color signal (G ), 
and a third basic color signal ( 5 ) from the digital signals; and 

correcting the basic color signals to obtain standardized signals, in which a 
three color signal matrix containing the first, second and third basic color signals ( /J , G,B) 
10 is multiplied by a correction matrix containing coefficients that depend on the analog 
preprocessing. 

In a first embodiment, the analog preprocessing includes a white balance 
adjustment. An advantage of applying the white balance method before the analog color 
signals are converted into at least one digital signal is that the quantization errors in at least 
15 one of the basic color signals is minimized. An advantage of this method and this camera is 
that the white balance method is applied to signals that are free from quantization errors, 
ultimately lowering the quantization errors in the fully processed signals. 

In a second embodiment, the horizontal sum values of the correction matrix 
are adjusted to one for horizontal sum values larger than one, with the analog preprocessing 
20 comprising a corresponding multiplication in the analog preprocessing to ensure to the 

overall processing multiplication remains the same as in the prior art. An advantage is that 
this multiplication leads to smaller quantization errors. An advantage is that the amplification 
is apart from being cursed with lower quantization errors, not further effected. 

These and other aspects of the invention will be apparent from, and elucidated 
25 with reference to, the embodiments described hereafter. 
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In the drawings: 

Fig. 1 shows an embodiment of the method in accordance with the first aspect 

of the present invention; and 
30 Fig. 2 shows a second embodiment of the method in accordance with the 

second aspect of the present invention. 

In both methods according to the invention for processing basic color signals 
in a color camera, light is collected and directed via a lens L into a sensor 1, preferably an 
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3 26.10.2000 
RGB Bayer sensor supplying a single signal in which the color signals are multiplexed in 
accordance with the color mask on the sensor (line-altematingly RGRG. . . and GBGB. . 
The signals correspond to a gain in the basic colors. According to a first aspect of the 
invention, as shown in Fig. 1, these signals are pre-processed according to a white balance 
5 method 3 and then submitted to an analog/digital converter (ADC) 5. The digital signal is 
then subjected to a reconstruction method carried out by the RGB reconstruction unit 7 that 
generates all 3 RGB signals for each pixel where the sensor output signal had only one of 
these RGB signals for each pixel. The reconstructed signals comprise a first basic color 
signal R, a second basic color signal G, and a third basic color signal B. These basic color 
10 signals need to be corrected to the EBU standardized signals to which television sets and PC 
monitors worldwide are accustomed to. This correction comprises multiplication of a three 
color signal matrix containing the first, the second, and the third basic color signal R, G, B by 
a correction matrix containing nine adjustable coefficients referred to as with X being 1, 

2 or 3 and y being 1,2 or 3. Mathematically this is expressed as follows: 



(1) 



wherein R^^G^ and are the output RGB signals and ^G^^B. the input RGB signals of 
the matrix 9. The standardized output signals , and B^ are then further processed as 

well as submitted to a white-balance-measurement- and-matrix-control-unit A. By this unit A 
the total contribution of each color measured over the whole scene, with the option that it is 

20 allowed to limit the number of pixels if they are distributed over that scene, is measured. To 
further illustrate the method according to the invention, the World Gray Assumption method 
is applied as an example of a white balance method. In the WGA method awfe/? is total 
contribution of Red/total contribution of Green and awbB is total contribution of Blue/total 
contribution of Green as measured/determined by unit A. These parameters owbR and OMfhB 

25 based on the standardized signals R^ , and B^ are used for regulation of the white 

balanced method. The white balanced method as applied to the analog basic color signals 
R^ , G^ and B^ involves then a multiplication with the parameters contained from the white- 

balance-measurement-and-matrix-control-unit A. Using the WGA method as the white 
balance method, the regulation would involve a multiplication of the basic color signals 
30 » G , B according to: 
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G , = G, (2) 

^awb a 

As a result, the white balance gains for red and blue are normalized to that of green. 
The input of the RGB reconstruction unit 7 is then: 
R\ = awbR X R^ 
G\ = awbR X G, 
Bi = awbB X 

where the prime refers to the fact that the signals have been subjected to the white balance 
method. 

After subjection to the correction method for standardization, i.e. after 
multiplication by the matrix ttie basic color signals follow: 
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It must be clear that the matrix coefficients must be adjusted from their values 
valid for the prior art situation in which the white balance method takes place after the matrix 
9 to settings which will be valid for die situation of Fig. 1 in which the white balance 3 takes 
place before the matrix 9. To this end the output of the white balance method is calculated for 
the situation in which the white balance method takes place after multiplication of the signals 
by the matiix. Assuming that the WGA method is appUed, the output would have been: 
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Comparing fonnulas 4 and 5 it is clear that the original mattix coefficients have to be 
adjusted to matrix coefficients b^ v/ithx being 1,2, 3 and y being 1,2, or 3, wherein, 
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(6) 



New values for the matrix coefficients are calculated by the amplifier control 
and matrix control unit. The calculation is ideally executed by a micro-controller. The 
5 method is, in that case, continuously applicable. At the output of the matrix 9 there may a 
ganmia correction and an RGB to YUV conversion, both not shown. 

Another method useful for reducing the quantization errors is shown in Fig. 2, 
according to another aspect of the invention. In this method the analog color signals are 
10 amplified by an amplifier 2. The analog color signals are to this end multiplied by a factor at 
least as large as 1 as further elaborated on in the following. To outline this method, the sum 
of the horizontal matrix coefficients is written as: 

XG^a^.+a^^ + a^ (7) 

15 More specifically the method involves a multiplication of the first, second and 

third analog basic color signal , and from the sensor 1 by the following factors: 
c/?, cG and cB . The values of c/?, cG and cB are determined by the following mles: 
if E/?>1 then cR = I^R else cR^l 

if ZG > 1 then cG = SG else cG = l (8) 
20 if ZS >1 then cB = 25 else cB = l 
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6 26.10.2000 
To maintain the original signal ampUtude, the reconstructed signals need to be subjected to a 
compensation for the ampUfication. The correction for standardization should therefore be as 
follows: 
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(9) 



It follows then that the original matrix coefficients xXy need to be replaced 
by coefficients with x being 1, 2 or 3 and y being 1,2 or 3, according to the following 
rules: 

b^^ = a„ / cR 
bij=a,JcB 
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(10) 



b^^=^a^,/cR 
b^=a^/cR 

^31 = ^31 I 

b,, = / CB 

where with x being 1,2,3 and y being 1,2, or 3, represent the new matrix coefficients. 
The white balance method 1 1 can now be appUed to the compensated reconstructed signals 
which are cursed with lower quantization errors, compared to the known method. The 
replacement of the matrix coefficients by the newly derived matrix coefficients only take 
15 place once, for instance when the camera is switched on. The replacement of a 

the regulation of the amplifier is carried out by the amplifier control & matrix control unit B. 

Salient aspects of the present invention include the idea of improving the 
digital quantization by shifting certain camera functions (white balance or default adjustment 
20 of the correction matrix) from the digital domain to the analog domain. The realization of the 
white balance on the multiplexed analog raw data from the RGB Bayer image sensor means 
that less quantization errors will occur in the red and blue color signals. For improving the 
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7 26.10.2000 
digital quantization, the matrix in the digital domain is preferably continuously corrected in 
case of a continuously changing white balance in the analog domain; hereby, the color 
reproduction is maintained in comparison with a full digitized method using first the matrix 
and thereafter the white balance. Alternatively, in the analog domain, the horizontal sum of 
S the correction matrix coefficients is adjusted in such a way, that in the digital domain, the 
horizontal sum is almost unity. The used method guarantees the maintenance of color 
reproduction and has to be executed only once, for example if the camera is switched on. 

Although embodiments of the present invention have been described in detail 
herein, it is to be understood that this invention is not limited to these precise embodiments 

10 and that modifications and variations may be effected therein by one skilled in the art without 
departing from the scope of the invention as defined by the appended claims. In the claims, 
any reference signs placed between parentheses shall not be construed as limiting the claim. 
The word "comprising" does not exclude the presence of elements or steps other than those 
listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of 

15 a plurality of such elements. The invention can be implemented by means of hardware 

comprising several distinct elements, and by means of a suitably programmed computer. In 
the device claim enumerating several means, several of these means can be embodied by one 
and the same item of hardware. The mere fact that certain measures are recited in mutually 
different dependent claims does not indicate that a combination of these measures cannot be 

20 used to advantage. 
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CLAIMS: 01 11. 

(g) 



1. A method of processing analog color signals, the method comprising: 
analog preprocessing (2, 3) sensor output signals to obtain analog 

preprocessed signals that cause a reduced amount of digital quantization errors; 

converting (3) the analog preprocessed signals into digital signals; 
5 reconstructing (7) a first basic color signal ( /? ), a second basic color signal 

( G ), and a third basic color signal (B) from the digital signals; and 

correcting (9) the basic color signals to obtain standardized signals, the 
correcting step comprising multiplication of a three color signal matrix containing the first, 
second and third basic color signals ( /? , G , B ) by a correction matrix containing coefficients 
10 that depend on the analog preprocessing step (2, 3). 

2. A method according to claim 1, wherein the analog preprocessing step 
includes a white balance adjustment. 

15 3. A method according to claim 2, wherein the coefficients of the correction 

matrix depend on the analog preprocessing step in that correction matrix coefficients aiQ 

replaced by coefficients with 
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(6) 



wherein awbR equals a total contribution of Red divided by a total contribution of Green 
20 and awbB equals a total contribution of Blue divided by a total contribution of Green 
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wherein the total contributions of Red, Green and Blue are determined from the standardized 
signals. 

4 A method according to claim 1 , wherein the sensor output signals comprise 

5 first, second and third analog color signals R, , G, and , and wherein said analog 

preprocessing step includes respectively multiplying the color signals by 

cR 
cG 
cB 

where cR^^R if else cR = l; 

where cG = Xg Sg>1 else cG = l; 

10 where cB = Yb if2B>l else cB = l, with 

^B = aji +032+033 

with being the coefficients the correction matrix would have without the analog 
preprocessing step, and wherein the coefficients of the correction matrix are replaced by 
coefficients with 
15 b^=a^lcR for x = 1,2,3 and ^ = 1; 
b„ = a^lcG for x = 1,2,3 and y = 2 ; 
b^^a^/cB for x = 1,2,3 and y = 3. 

5 A device for processing analog color signals, the device comprising: 

20 means for analog preprocessing (2, 3) sensor output signals to obtain analog 

preprocessed signals that cause a reduced amount of digital quantization errors; 

means for converting (5) the analog preprocessed signals into digital signals; 
means for reconstructing (7) a first basic color signal ( if ), a second basic 
color signal ( G ), and a third basic color signal ( B ) from tiie digital signals; and 
25 means for correcting (9) the basic color signals to obtain standardized signals, 

the correcting means comprising means for multiplying a three color signal matrix containing 
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10 26.10.2000 
the first, second and third basic color signals ( /? , G , 5 ) by a correction matrix containing 
coefficients that depend on the analog preprocessing means (2, 3). 

6. A device according to claim 5, wherein the analog preprocessing means (3) 

includes means (3) for carrying out a white balance adjustment. 



15 



20 



7. A method according to claim 6, wherein the coefficients of the correction 

-matrix depend on the analog preprocessing step in that correction matrix coefficients are 

replaced by coefficients with 
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&33 — ^33 

wherein awbR equals a total contribution of Red divided by a total contribution of Green 
and awbB equals a total contribution of Blue divided by a total contribution of Green 
wherein the total contributions of Red, Green and Blue are determined from the standardized 
signals. 

8. A device according to claim S, wherein the sensor output signals comprise 

first, second and third analog color signals , and , and wherein said analog 
preprocessing means (2) includes means (2) for respectively multiplying the color signals by 
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2/? = a„ +ai2 +ai3 

with being the coefficients the correction matrix would have without the analog 
preprocessing step, and wherein the coefficients fl^of the correction matrix arc replaced by 

coefficients with 

5 b^=a^/cR for x = 1,2:3 and y = l; 

= a^/cG for x = 1,2,3 and y = 2; 

b^=a^/cB for ;c = 1,2,3 and y = 3, 

9. A color camera comprising: 

10 a sensor for generating sensor output signals; and 

a device as claimed in claim 5. 
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A method and device for processing analog color signals, comprising: 

analog preprocessing (3) sensor output signals to obtain analog preprocessed 
signals that cause a reduced amount of digital quantization errors; 

converting (5) the analog preprocessed signals into digital signals; 

reconstructing (7) a first basic color signal ( /? ), a second basic color signal 
( G ), and a third basic color signal ( 5 ) from the digital signals; and 

correcting (9) the basic color signals to obtain standardized signals, in which a 
three color signal matrix containing the first, second and third basic color signals ( /? , G ,B) 
is multiplied by a correction matrix containing coefficients that depend on the analog 
preprocessing (3). 

In a first embodiment, the analog preprocessing includes a white balance 
adjustment. In a second embodiment, the horizontal sum values of the correction matrix are 
adjusted to one for horizontal sum values larger than one, with the analog preprocessing 
comprising a corresponding multiplication in the analog preprocessing to ensure to the 
overall processing multiplication remains the same as in the prior art. 

The invention further relates to a color camera comprising an RGB Bayer 
sensor for generating the sensor output signals, and the above-mentioned processing device. 



(Fig. 1) 



THIS PAGE BLAMSC (uspto 



i 



